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Abstract 

CO 

O Reservoir computing is a new, powerful and flexible machine learn- 

ing technique that is easily implemented in hardware. Recently, by us- 
I ing a time-multiplexed architecture, hardware reservoir computers have 

reached performance comparable to digital implementations. Operating 
speeds allowing for real time information operation have been reached 
using optoelectronic systems. At present the main performance bottle- 
y—{ neck is the readout layer which uses slow, digital postprocessing. We have 

C*") designed an analog readout suitable for time-multiplexed optoelectronic 

reservoir computers, capable of working in real time. The readout has 
been built and tested experimentally on a standard benchmark task. Its 
performance is better than non-reservoir methods, with ample room for 
t-H further improvement. The present work thereby overcomes one of the 

major limitations for the future development of hardware reservoir com- 
puters. 



1 Introduction 



The term "reservoir computing" encompasses a range of similar machine learning 
techniques, independently introduced by H. Jaeger [T] and by W. Maass [2J. 
While these techniques differ in implementation details, they share the same 
core idea: that one can leverage the dynamics of a recurrent nonlinear network 
to perform computation on a time dependent signal without having to train 
the network itself. This is done simply by adding an external, generally linear 
readout layer and training it instead. The result is a powerful system that 
can outperform other techniques on a range of tasks (see for example the ones 
reported in (3j H]), and is significantly easier to train than recurrent neural 
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networks. Furthermore it can be quite easily implemented in hardware [5J [HI E], 
although it is only recently that hardware implementations with performance 
comparable to digital implementations have been reported [8j [9j [10] . 

One great advantage of this technique is that it places almost no require- 
ments on the structure of the recurrent nonlinear network. The topology of the 
network, as well as the characteristics of the nonlinear nodes, are left to the user. 
The only requirements are that the network should be of sufficiently high dimen- 
sionality, and that it should have suitable rich dynamics. The last requirement 
essentially means that the dynamics allows the exploration of a large number 
of network states when new inputs come in, while at the same time retaining 
for a finite time information on the previous inputs For this reason, the 

reservoir computers appearing in literature use widely different nonlinear units, 
see for example |TJ [2 [5J [TS] and in particular the time multiplexing architecture 
proposed in 00 HQ]. 

Optical reservoir computers are particularly promising, as they can provide 
an alternative path to optical computing. They could leverage the inherent high 
speeds and parallelism granted by optics, without the need for strong nonlinear 
interaction needed to mimic traditional electronic components. Very recently, 
optoelectronic reservoir computers have been demonstrated by different research 
teams [T01I5], conjugating good computational performances with the promise of 
very high operating speeds. However, one major drawback in these experiments, 
as well as all preceding ones, was the absence of readout mechanisms: reservoir 
states were collected on a computer and post-processed digitally, severely lim- 
iting the processing speeds obtained and hence the applicability. 

An analog readout for experimental reservoirs would remove this major bot- 
tleneck, as pointed out in |13| . The modular characteristics of reservoir comput- 
ing imply that hardware reservoirs and readouts can be optimized independently 
and in parallel. Moreover, an analog readout opens the possibility of feeding 
back the output of the reservoir into the reservoir itself, which in turn allows 
the use of different training techniques [H] and to apply reservoir computing to 
new categories of tasks, such as pattern generation |15l IT5] . 

In this paper we present a proposal for the readout mechanism for opto- 
electronic reservoirs, using an optoelectronic intensity modulator. The design 
that we propose will drastically cut down their operation time, specially in the 
case of long input sequences. Our proposal is suited to optoelectronic or all- 
optical reservoirs, but the concept can be easily extended to any experimental 
time-multiplexed reservoir computer. The mechanism has been tested experi- 
mentally using the experimental reservoir reported in [10 , and compared to a 
digital readout. Although the results are preliminary, they are promising: while 
not as good as those reported in (TU], they are however already better than 
non-reservoir methods for the same task |16| . 



2 



2 Reservoir computing and time multiplexing 



2.1 Principles of Reservoir Computing 

The main component of a reservoir computer (RC) is a recurrent network of 
nonlinear elements, usually called "nodes" or "neurons". The system typically 
works in discrete time, and the state of each node at each time step is a function 
of the input value at that time step and of the states of neighboring nodes at 
the previous time step. The network output is generated by a readout layer - a 
set of linear nodes that provide a linear combination of the instantaneous node 
states with fixed coefficients. 

The equation that describes the evolution of the reservoir computer is 



where Xi(n) is the state of the i-th node at discrete time n, N is the total 
number of nodes, u(n) is the reservoir input at time n, nii and are the 
connection coefficients that describe the network topology, a and /3 are two 
parameters that regulate the network's dynamics, and / is a nonlinear function. 
One generally tunes a and /3 to have favorable dynamics when the input to be 
treated is injected in the reservoir. The network output y{n) is then constructed 
using a set of readout weights Wi and a bias weight Wb, as 



Training a reservoir computer only involves the readout layer, and consists 
in finding the best set of readout weights Wi and bias Wb that minimize the 
error between the desired output and the actual network output. Unlike con- 
ventional recurrent neural networks, the strength of connections mi and tiiq are 
left untouched. As the output layer is made only of linear units, given the full 
set of reservoir states Xj(n) for all the time steps n, the training procedure is a 
basic, regularized linear regression. 

2.2 Time multiplexing 

The number of nodes in a reservoir computer determines an upper limit to the 
reservoir performance |17j : this can be an obstacle when designing physical im- 
plementations of RCs, which should contain a high number of interconnected 
nonlinear units. A solution to this problem proposed in [5], is time multiplex- 
ing: the Xi(n) are computed one by one by a single nonlinear element, which 
receives a combination of the input u(n) and a previous state Xj(n — 1). In 
addition an input mask mj is applied to the input u(n), to enrich the reservoir 
dynamics. The value of Xi(n) is then stored in a delay line to be used at a later 
time step n + 1. The interaction between different neurons can be provided 
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Figure 1: Scheme of the experimental setup, including the optoelectronic reser- 
voir ('Input' and 'Reservoir' layers) and the analog readout ('Output' layer). 
The red and green parts represent respectively the optical and electronic com- 
ponents. "AWG": Arbitrary waveform generator. "M-Z": LiNbO^ Mach-Zehnder 
modulator. "FPD": Feedback photodiode. "AMP": Amplifier. "Scope": NI PXI 
acquisition card. 



by either having a slow nonlinear element which couples state Xi to the previ- 
ous states Xi—i,Xi—2, ... |B], or by using an instantaneous nonlinear element and 
desynchronizing the input with respect to the delay line |10| . 

2.3 Hardware RC with digital readout 

The hardware reservoir computer we use in the present work is identical to the 
one reported in [TU] (see also [OJ). It uses the time- multiplexing with desyn- 
chronisation technique described in the previous paragraph. We give a brief 
description of the experimental system, represented in the left part of Figure 
[I] It uses a LiNbOz Mach-Zehnder (MZ) modulator, operating on a constant 
power 1560 nm laser, as the nonlinear component. A MZ modulator is a voltage 
controlled optoelectronic device; the amount of light that it transmits is a sine 
function of the voltage applied to it. The resulting state Xi(n) is encoded in a 
light intensity level at the MZ output. It is then stored in a spool of optical 
fiber, acting as delay line of duration T = 8.5/is, while all the subsequent states 
Xi(n) are being computed by the MZ modulator. When a state Xi(n) reaches 
the end of the fiber spool it is converted into a voltage by a photodiode. 

The input u(n) is multiplied by the input mask rrij and encoded in a voltage 
level by an Arbitrary Waveform Generator (AWG). The two voltages corre- 
sponding to the state Xi(n) at the end of the fiber spool and the input m,iu(n) 
are added, amplified, and the resulting voltage is used to drive the MZ modu- 
lator, thereby producing the state xAn + 1), and so on for all values of n. 

In the experiment reported in [10j a portion of the light coming out of the 
MZ is deviated to a second photodiode (not shown in Figure [I]), that converts it 
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into a voltage and sends it to a digital oscilloscope. The Mach-Zehnder output 
can be represented as "steps" of light intensities of duration 9 (see Figure [2ji) , 
each one representing the value of a single node state Xi at discrete time n. The 
value of each Xi(n) is recovered by taking an average of the measured voltage 
for each state at each time step. The optimal readout weights Wj and bias Wb 
are then calculated on a computer from a subset (training set) of the recorded 
states, using ridge regression [T5], and the output y(n) is then calculated using 
equation [2] for all the states collected. The performance of the reservoir is then 
calculated by comparing the reservoir output y(n) with the desired output y(n). 



3 Analog readout 
Readout scheme 

Developing an analog readout for the reservoir computer described in section [2] 
means designing a device that multiplies the reservoir states shown in Figure [2^t, 
by the readout weights Wi, and that sums them together in such a way that the 
reservoir output y(n) can be retrieved directly from its output. However, this 
is not straightforward to do, since obtaining good performance requires positive 
and negative readout weights Wi. In optical implementations |10[ 19] the states 
Xi are encoded as light intensities which are always positive, so they cannot be 
subtracted one from another. Moreover, the summation over the states must 
include only the values of Xi pertaining to the same discrete time step n and 
reject all other values. This is difficult in time-multiplexed reservoirs, where the 
states XAr(n) and x\{n + 1) follow seamlessly. 

Here we show how to resolve both difficulties using the scheme depicted in 
the right panel of Figure [T] Reservoir states encoded as light intensities in the 
optical reservoir computer and represented in Figure [2^, are fed to the input of 
a second MZ modulator with two outputs. A second function generator governs 
the bias of the second Mach-Zehnder, providing the modulation voltage V(t). 
The modulation voltage controls how much of the input light passing through 
the readout Mach-Zehnder is sent to each output, keeping constant the sum of 
the two output intensities. The two outputs are connected to the two inputs of a 
balanced photodiode, which in turn gives as output a voltage level proportional 
to the difference of the light intensities received at its two input^j] This allows 
us to multiply the reservoir states by both positive and negative weights. 

The time average of the output voltage of the photodiode is obtained by using 
a capacitor. The characteristic time of the analog integrator r is proportional to 
the capacity The role of this time scale is to include in the readout output 
all the pertinent contributions and exclude the others. The final output of the 

A balanced photodiode consists of two photodiodes which convert the two light intensities 
into two electric currents, followed by an electronic circuit which produces as output a voltage 
proportional to the difference of the two currents 

2 In the case where the impedance of the coaxial cable R = 50f2 is matched with the output 
impedance of the photodiode, we have r = 
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reservoir is the voltage across the capacitor at the end of each discretized time 
n. 

What follows is a detailed description of the readout design. 



Multiplication by arbitrary weights 

The multiplication of the reservoir states by arbitrary weights, positive or neg- 
ative, is realized by the second MZ modulator followed by the balanced pho- 
todiode. The modulation voltage V(t) that drives the second Mach Zehnder is 
piecewise constant, with a step duration equal to the duration 9 of the reservoir 
states; transitions in voltages and in reservoir states are synchronized. The mod- 
ulation voltage is also a periodic function of period ON, so that each reservoir 
state Xi{n) is paired with a voltage level V{ that doesn't depend on n. The light 
intensities 0\ (i) and O2 (t) at the two outputs of the Mach-Zehnder modulator 
are 

^ M _ TM 1 + COS((V(t) + V Mas )^ + <p) „ _ 1 - COs((V(t) + V bl as)^ + <f) 
L>i{t) - l(t) ,0 2 (t) - l{t) , 

(3) 

where I(t) is the light intensity coming from the reservoir, Vuas is a con- 
stant voltage that drives the modulator, ip is an arbitrary, constant phase value, 
and is the half-wave voltage of the modulator. Neglecting the effect of any 
bandpass filter in the photodiode, and choosing Vbi as appropriately, the output 
P(t) from the photodiode can be written as 

P{t) = G{O x {t) - 2 (t)) = J(t)(Gsin(^)) = I(t)W(t) (4) 

with G a constant gain factor. In other words, by setting the right bias and 
driving the modulator with a voltage V(t), we multiply the signal I(t) by an 
arbitrary coefficient W(t). Note that, if V(t) is piecewise constant, then W(t) is 
as well. This allows us to achieve the multiplication of the states Xi(n), encoded 
in the light intensity I(t), by the weights Wj, just by choosing the right voltage 
V(t), as shown in Figure ph. 



Summation of weighted states 

To achieve the summation over all the states pertaining to the same discrete 
time step n, which according to equation [2] will give us the reservoir output 
minus the bias Wb, we use the capacitor at the right side of the Output layer in 
Figure^ The capacitor provides the integration of the photodiode output given 
by eq. |4] with an exponential kernel and time constant r. If r is significantly 
less than the amount of time ON needed for the system to process all the nodes 
relative to a single time step, we can minimize the crosstalk between node states 
relative to different time steps. 

Let us consider the input I(t) of the readout, and let t = be the instant 
where the state of the first node for a given discrete time step n begins to 
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Figure 2: a) Reservoir output I(t). The gray line represents the output as 
measured by a photodiode and an oscilloscope. We indicated for reference the 
time = 130ns used to process a single node and the duration ON — 8.36/is 
of the whole set of states, b) Output P(t) of the balanced photodiode (see 
equation with the trace of panel a) as input, before integration, c) Voltage 
Q(t) on the capacitor for the same input (see equation [5]) . The integration time 
r is indicated for reference. The black dots indicate the values at the end of 
each discretized time n, taken as the output y(n)oi the analog readout. 



be encoded in I(t) . Using equation [4] we can write the voltage Q(t) on the 
capacitor at time ON as 

Q(0N) = Q(0)e~^ + / 7(s)TU(s)e~^ds (5) 
Jo 

For < t < ON, we have 

I(t) = Xi(n), W{t) = Wi, for 9(i - 1) < t < Oi (6) 
Integrating equation [5] yields 

N 

6N x ^ 8(N — i) ft 

Q{6N) = Q(0)e-— + y £ / x i (n)r H w u m = e (l-e~ )t (7) 

i=l 

Equation [7] shows that, at time 0iV, the voltage on the capacitor is a linear 
combination of the reservoir states for the discrete time n, with node-dependent 
coefficients t^w.;, plus a residual of the voltage at time 0, multiplied by an 
extinction coefficient e~~. At time 20 N the voltage on the capacitor would 
be a linear combination of the states for discrete time n + 1, multiplied by the 
same coefficients, plus a residual of the voltage at time ON, and so on for all 
values of n and corresponding multiples of ON. 

A simple procedure would encode the weights uii = ^p- onto the voltage V(t) 
that drives the modulator , provide an external, constant bias Wb, and have 
the output y(n) of the reservoir, defined by equation [2] effectively encoded on 
the capacitor. This simple procedure would however be unsatisfactory because 
unavoidably some of the r\i would be very small, and therefore the Wi would be 
large, spanning several orders of magnitude. This is undesirable, as it requires 
a very precise control of the modulation voltage V(t) in order to recreate all the 
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Wi values, leaving the system vulnerable to noise and to any non-ideal behavior 
of the modulator itself. 

To mitigate this, we adapt the training algorithm based on ridge regression 
to our case. We redefine the reservoir states as £j(n) = Xi{n)rji\ we then calculate 
the weights u>i that, applied to the states £j, give the best approximation to the 
desired output y(n). The advantage here is that ridge regression keeps the norm 
of the weight vector to a minimum; by redefining the states, we can take the 
rji into account without having big values of Wi that force us to be extremely 
precise in generating the readout weights. 

A sample trace of the voltage on the capacitor is shown in Figure [2J;. 

Hardware implementation 

To implement the analog readout, we started from the experimental architec- 
ture described in Section [2] and we added the components depicted in the right 
part of Figure [l] For the weight multiplication, we used a second Mach-Zehnder 
modulator (Photline model MXDO-LN-10 with bandwidth in excess of 10GHz 
and — 5.9V), driven by a Tabor 2074 Arbitrary Waveform Generator (max- 
imum sampling rate 200 MSamples/s). The two outputs of the modulator were 
fed into a balanced photodiode (Terahertz technologies model 527 InGaAs bal- 
anced photodiode, bandwidth set to 125MHz, response set to 1000V/W), whose 
output was read by the National Instruments PXI digital acquisition card (sam- 
pling rate 200 MSamples/s). 

In most of the experimental results described here, the capacitor at the 
end of the circuit was simulated and not physically inserted into the circuit: 
this allowed us to quickly cycle in our experiments through different values of r 
without taking apart the circuit every time. The external bias Wb to the output, 
introduced in equation [2] was also provided after the readout. The reasoning 
behind these choices is that both these implementations are straightforward, 
while the use of a modulator and a balanced photodiode as a weight generator 
is more complex: we chose to focus on the latter issue for now, as our goal is to 
validate the proposed architecture. 

4 Results 

As a benchmark for our analog readout, we use a wireless channel equalization 
task, introduced in 1994 [19_ to test adaptive bilinear filtering and subsequently 
used by Jaeger [16] to show the capabilities of reservoir computing. This task 
is becoming a standard benchmark task in the reservoir computing community, 
and has been used for example in [20 . It consists in recovering a sequence of 
symbols transmitted along a wireless channel, in presence of multiple reflections, 
noise and nonlinear distortion; a more detailed description of the task can be 
found in the Appendix. The performance of the reservoir is usually measured in 
Symbol Error Rate (SER), i.e. the rate of misinterpreted symbols, as a function 
of the amount of noise in the wireless channel. 
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Figure 3: Performance of the analog readout. Left: Performance as a function 
of the input SNR, for a reservoir of 28 nodes, with t/6N = 0.18. Middle: 
Performance for the same task, for a reservoir of 64 nodes, r /ON — 0.18. Right: 
Performance as a function of the ratio t/0N , at constant input noise level (28 
dB SNR) for a reservoir of 64 nodes. The performance is measured in Signal 
Error Rate (SER). Blue triangles: reservoir with digital readout. Red squares: 
reservoir with ideal analog readout. Black circles: reservoir with experimental 
analog readout (simulated capacitor). Purple stars in the left panel: reservoir 
where a physical capacitor has been used. 



Figure [3] shows the performance of the experimental setup of |10j for a net- 
work of 28 nodes and one of 64 nodes, for different amounts of noise. For each 
noise level, three quantities are presented. The first is the performance of the 
reservoir with a digital readout (blue triangles), identical to the one used in |10J . 
The second is the performance of a simulated, ideal analog readout, which takes 
into account the effect of the rji coefficients introduced in equation [7] but no 
other imperfection. It produces as output the discrete sum Wf, + Yli=i ( rec ^ 
squares). This is, roughly speaking, the goal performance for our experimental 
readout. The third and most important is the performance of the reservoir as 
calculated on real data taken from the analog reservoir with the analog output, 
with the effect of the continuous capacitive integration computed in simulation 
(black circles) . 

As can be seen from the figure, the performance of the analog readout is fairly 
close to its ideal value, although it is significantly worse than the performance of 
the digital readout. However, it is already better than the non-reservoir methods 
reported in |19| and used by Jaeger as benchmarks in |16j . It can also handle 
higher signal-to-noise ratios. As expected, networks with more nodes have better 
performance; it should be noted, however, that in experimental reservoirs the 
number of nodes cannot be raised over a certain threshold. The reason is that 
the total loop time 9N is determined by the experimental hardware (specifically, 
the length of the delay line); as TV increases, the length 9 of each node must 
decrease. This leaves the experiment vulnerable to noise and bandpass effect, 
that may lead, for example, to an incorrect discretization of the Xi(n) values, 
and an overall worse performance. 

We did test our readout with a 70nF capacitor, with a network of 28 nodes, 
to prove that the physical implementation of our concept is feasible: the per- 
formance of this setup is shown in the left panel of Figure [3] The results are 
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comparable to those obtained in simulation, even if, at low levels of noise in the 
input, the performance of the physical setup is slightly worse. 

The rightmost panel of figure [3] shows the effects of the choice of the capacitor 
at the end of the circuit, and therefore of the value of r. The plot represents the 
performance at 28 dB SNR for a network of 64 nodes, for different values of the 
ratio t/ON, obtained by averaging the results of 10 tests. It is clear that the 
choice of r has a complicated effect on the readout performance; however, some 
general rules may be inferred. Too small values of r mean that the contribution 
from the very first nodes is vanishingly small, effectively decreasing the reservoir 
dimensionality, which has a strong impact on the performance both of the ideal 
and the experimental reservoir. On the other hand, larger values of t impact 
the performance of the experimental readout, as the residual term in equation [7] 
gets larger. A compromise value of t/0N = 0.222 seems to give the best result, 
corresponding in our case to a capacity of about 70 nF. 

5 Discussion 

To our knowledge, the system presented here is the first analog readout for an 
experimental reservoir computer. While the results presented here are prelim- 
inary, and there is much optimization of experimental parameters to be done, 
the system already outperforms non-reservoir methods. We expect to extend 
easily this approach to different tasks, already studied in [21110]! including a 
spoken digit recognition task on a standard datasetj^] 

Further performance improvements can reasonably be expected from fine- 
tuning of the training parameters: for instance the amount of regularization 
in the ridge regression procedure, that here is left constant at 1 • 10~ 4 , should 
be tuned for best performance. Adaptive training algorithms, such as the ones 
mentioned in [3T], could also take into account nonidealities in the readout 
components. Moreover the choice of r, as Figure [3] shows, is not obvious and a 
more extensive investigation could lead to better performance. 

The architecture proposed here is simple and quite straightforward to re- 
alize. It is very modular, meaning that it can be added at the output of any 
preexisting time multiplexing reservoir with minimal effort, whether it is based 
on optics or electronics. The capacitor at the end of the circuit could probably 
be substituted with a more complicated, active electronic circuit performing the 
summation of the incoming signal before resetting itself. This would eliminate 
the problem of residual voltages, and allow better performance at the cost of 
increased complexity of the readout. 

The main interest of the analog readout is that it allows optoelectronic reser- 
voir computers to fully leverage their main characteristic, which is the speed of 
operation. Indeed, removing the need for slow, offline postprocessing is indi- 
cated in [T3] as one of the major challenges in the field. Once the training is 
finished, optoelectronic reservoirs can process millions of nonlinear nodes per 

3 Texas Instruments-Developed 46- Word Speaker-Dependent Isolated Word Corpus (TI46), 
September 1991, NIST Speech Disc 7-1.1 (1 disc) (1991). 
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second [TO]; however, in the case of a digital readout, the node states must be 
recovered and postprocessed to obtain the reservoir outputs. It takes around 
1.6 seconds for the digital readout in our setup to retrieve and digitize the states 
generated by a 9000 symbol input sequence. The analog readout removes the 
need for postprocessing, and can work at a rate of about 8.5 /is per input sym- 
bol, five orders of magnitude faster than the electronic reservoir reported in 
0. 

Finally, having an analog readout opens the possibility of feedback - using 
the output of the reservoir as input or part of an input for the successive time 
steps. This opens the way for different tasks to be performed [T5] or different 
training techniques to be employed [14 . 
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Appendix: Nonlinear Channel Equalization task 

What follows is a detailed description of the channel equalization task used 
to test the reservoir performance. The goal for this task is to reconstruct a 
sequence d(n) of symbols taken from {—3,-1,1,3}. The symbols in d(n) are 
mixed together in a new sequence q(n) given by 

q(n) = 0.08d(n + 2)-0.12d(n+l) + d(n)+0.18d(n-l)-0.1d(n-2) (8) 
+0.091d(n - 3)-0.05<i(n - 4) + 0.04d(n - 5) + 0.03d(n - 6) + 0.0W(n-7) 

which models a wireless signal reaching a receiver through different paths with 
different traveling times. A noisy, distorted version u(n) of the mixed signal q(n), 
simulating the nonlinearities and the noise sources in the receiver, is created by 
having u(n) = q(n) + 0.036q(n) 2 — 0.011q(n) 3 + v(n), where v{n) is an i.i.d. 
Gaussian noise with zero mean adjusted in power to yield signal-to-noise ratios 
ranging from 12 to 32 dB. The sequence u(n) is then fed to the reservoir as an 
input; the output of the readout R(n) is rounded off to the closest value among 
{—3, —1, 1, 3}, and then compared to the desired symbol d(n). The performance 
is usually measured in Signal Error Rate (SER), or the rate of misinterpreted 
symbols. 
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